• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    It is a method of identifying a tracked object that has a known database of spatial and hyperspectral information. The method associates an identifier with the tracked object. selects a parameter associated with the spatial or hyperspectral information of the tracked object; detects a deviation in the selected parameter; compares the deviation with the database, and if the deviation exceeds a predetermined threshold, assigns a new identifier to the scanned object, and if the deviation does not exceed the predetermined threshold, continues tracking of the scanned object.
    公开号:BR102013020974A2
    申请号:R102013020974-0
    申请日:2013-08-16
    公开日:2018-02-14
    发明作者:Daniel Buehler Eric；Thomas Occhipinti Benjamin；Robert Kuczynski Konrad
    申请人:Ge Aviation Systems Llc；
    IPC主号:
    专利说明:

    (54) Title: METHOD OF IDENTIFICATION OF A TRACKED OBJECT (51) Int. Cl .: G06K 9/00 (52) CPC: G06K 9/00 (30) Unionist Priority: 17/08/2012 US 13 / 588,568 (73 ) Holder (s): GE AVIATION SYSTEMS LLC (72) Inventor (s): ERIC DANIEL BUEHLER; BENJAMIN THOMAS OCCHIPINTI; KONRAD ROBERT KUCZYNSKI (74) Attorney (s): ARTUR FRANCISCO SCHAAL (57) Abstract: This is a method of identifying a tracked object that has a known database of spatial and hyperspectral information. The method associates an identifier with the tracked object; selects a parameter associated with the spatial or hyperspectral information of the tracked object; detects a deviation in the selected parameter; compares the deviation with the database, and, if the deviation exceeds a predetermined threshold, assigns a new identifier to the tracked object, and, if the deviation does not exceed the predetermined threshold, the tracking of the tracked object continues.
    Fig. 1
    1/23 “METHOD OF IDENTIFYING A TRACKED OBJECT” Field of the Invention [001] The invention is located in the fields of data processing, more specifically in the field of image processing. The invention relates to a method of identifying a tracked object.
    Background of the Invention [002] The environment of a remote detection system for hyperspectral images (HSI) is well described in Hyperspectral Image Processing for Automatic Target Detection Applications by Manolakis, D., Marden, D. and Shaw G. (Journal Lincoln Laboratory ; Volume 14; 2003 pages. 79 to 82). An imaging sensor has pixels that record a measurement of hyperspectral energy. An HSI device will record energy in a matrix of pixels that capture spatial information by the geometry of the matrix and capture spectral information by taking measurements on each pixel of a number of continuous hyperspectral bands. The processing of spatial and spectral information still depends on a specific application of the remote detection system.
    [003] The remotely detected HSI has proven to be useful for broad applications that include monitoring with terrestrial and environmental use and military reconnaissance and surveillance. HSI provides image data that contains both spectral and spatial information. These types of information can be used for remote detection and job tracking. Specifically, given a set of visual sensors mounted on a platform such as an unmanned aerial vehicle (UAV) or ground station, an HSI video can be acquired and a set of algorithms can be applied to the spectral video to detect and track frame objects frame.
    [004] Spectral based processing algorithms have been developed to classify or group similar pixels; that is, pixels
    Petition 870160036329, of 7/14/2016, p. 16/39
    2/23 with signatures or similar spectral characteristics. Processing in this manner alone is not susceptible to target detection and tracking applications in which the number and size of targets in a situation are usually too small to support the estimation of the statistical properties needed to classify the type of target. However, the spatial processing of the typical HSI is adjusted by the low spatial resolution of the typical systems that collect the HSI. As a result, remote capture systems that collect and process HSI are typically developed due to a trade-off between spatial and spectral resolution to maximize detection of both resolved and unresolved targets, where a resolved target is an object imagined by more than a pixel. In this mode, spectral techniques can detect targets not resolved by their spatial and signature techniques that can detect targets resolved by their shape.
    [005] Numerous hyperspectral search algorithms have been developed and used in the processing of HSI for the purpose of target detection. These hyperspectral search algorithms are usually designed to explore statistical characteristics of candidate targets in the images and are usually built on known statistical concepts. For example, the Mahalanobis distance is a statistical measure of the similarity that was applied to the hyperspectral pixel signatures. The Mahalanobis distance measures signature similarity by testing the signature against a mobile, standard deviation from a known class of signatures.
    [006] Other known techniques include Spectral Angle Mapping (SAM), Spectral Information Divergence (SID), Zero Differential Area (ZMDA) and Bhattacharyya distance. SAM is a method for comparing a target signature candidate to a signature known for treating each spectrum as vectors and calculating the angle between the vectors. Due to SAM using only the vector direction and not the vehicle length 870160036329, from 07/14/2016, p. 17/39
    3/23 tor, the method is insensitive to variation in lighting. The SID is a method to compare a candidate target signature to a signature known for measuring the probabilistic divergence or discrepancy between the spectrum. The ZMDA normalizes the candidate target and the signatures known for their variance and computes their difference, which corresponds to the area between the two vectors. The Bhattacharyya distance is similar to the Mahalanobois distance, but it is used to measure the distance between a set of candidate target signatures against a known class of signatures.
    Description of the Invention [007] The invention relates to a method of identifying a tracked object. The method comprises tracking an object with the use of an imaging sensor, in which the tracked object has a known database of spatial and hyperspectral information; associate an identifier to the tracked object; select at least one parameter associated with the spatial or hyperspectral information of the tracked object; detect a deviation in at least one selected parameter; compare the deviation with the database; and if the deviation exceeds a predetermined threshold, it assigns a new identifier to the tracked object, and, if the deviation does not exceed the predetermined threshold, the tracking of the tracked object continues.
    Brief Description of the Drawings [008] Figure 1 is a diagrammatic view of a method for tracking and determining a probability of detecting the objects observed in the HSI according to a first embodiment of the invention.
    [009] Figure 2 is a diagrammatic view of a method for selecting a hyperspectral search algorithm according to an embodiment of the invention.
    Petition 870160036329, of 7/14/2016, p. 18/39
    4/23 [010] Figure 3 is a diagrammatic view of a method for selecting a tolerance for a hyperspectral search algorithm according to an embodiment of the invention.
    [011] Figure 4a shows a situation in which a hyperspectral imaging system in accordance with an embodiment of the invention has detected and tracked two objects.
    [012] Figure 4b shows a situation in which a hyperspectral imaging system according to an embodiment of the invention detects changes in the scanned objects.
    Description of Realizations of the Invention [013] In the background and in the description below, for explanatory purposes, the numerous specific details are defined in order to provide a complete understanding of the technology described in this document. It will be apparent to a person skilled in the art, however, that exemplary achievements can be practiced without these specific details. In other cases, the structures and device are shown in diagram form in order to facilitate the description of the exemplary achievements.
    [014] Exemplary achievements are described with reference to the drawings. These drawings illustrate certain details of the specific achievements that implant a module, computer program product or method described in this document. However, the drawings should not be interpreted as imposing any limitations that may be present in the drawings. The computer program product and method can be provided in any machine-readable medium to complete its operations. Achievements can be deployed using an existing computer processor or by a special purpose computer processor incorporated for this or other purpose or by a programmed system.
    Petition 870160036329, of 7/14/2016, p. 19/39
    5/23 [015] As noted above, the achievements described in this document may include a computer program product that comprises the machine-readable medium for transporting or having machine-executable instructions or data structures stored therein. Such a machine-readable medium can be any medium available, which can be accessed by a special-purpose or general-purpose computer or another machine with a processor. By way of example, such a machine-readable medium may comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices or any other means that can be used to transport or store the desired program code in the form of instructions executable by machine or data structures and which can be accessed by a special purpose or general purpose computer or another machine with a processor. When information is transferred or provided over a network or other communication connection (even programmed, wireless or a programmed or wireless match) to a machine, the machine correctly views the connection as a machine-readable medium. Thus, any connection is correctly called a machine-readable medium. The combinations of the above are also included within the scope of the machine-readable medium. Machine-executable instructions comprise, for example, instructions and data, which cause a general-purpose computer, special-purpose computer, or processing machines with a special purpose to perform a certain function or group of functions.
    [016] Achievements will be described in the general context of method steps that may be implemented in an implementation by a program product that includes instructions executable by machine, such as program code, for example, in the form of program modules executed by
    Petition 870160036329, of 7/14/2016, p. 20/39
    6/23 machines in network environments. In general, program modules include routines, programs, objects, components, data structures, etc. that have the technical effect of performing particular tasks or deploying particular abstract data types. The machine executable instructions, associated data structures and program modules represent examples of the program code for performing steps of the method disclosed in this document. The particular sequence of such associated data structures or executable instructions represents examples of the corresponding acts to implement the functions described in such steps.
    [017] Achievements can be practiced in a network environment with the use of logical connections to one or more remote computers that have processors. Logical connections can include a local area network (LAN) and an extended area network (WAN) that are present in the document by way of example and not limiting. Such network environments are common in computer networks that span the entire company or the entire office, intranets and the internet and can use a wide variety of different communication protocols. That person skilled in the art will find that such network communication environments will normally cover many types of computer system configuration, which include personal computers, handheld devices, multiprocessor systems, programmable or microprocessor based consumer electronics, network, minicomputers, mainframe computers and the like.
    [018] Achievements can also be practiced in distributed computing environments where tasks are performed by remote or local processing devices that are linked by link (even by programmed links, wireless links or by a correspondence of wireless links or programmed) through a communication network. In a distributed computing environment, the program modules can be located in the 870160036329, of 7/14/2016, p. 21/39
    7/23 located on both remote and local memory storage devices.
    [019] An exemplary system for deploying all or parts of the exemplary achievements needs to include a general purpose computing device in the form of a computer, which includes a processing unit, a system memory and a system bus that couple various components of systems that include system memory for the processing unit. System memory can include read-only memory (ROM) and random access memory (RAM). The computer can also include a magnetic hard drive for reading and writing to a magnetic hard disk, a magnetic disk drive for reading or writing to a removable magnetic disk, and a disk drive optical drive for reading a removable optical disc or writing to it as a CD-ROM or other optical medium. The drives and their associated machine-readable medium provide non-volatile storage of machine-executable instructions, data structures, program modules and other data to the computer.
    [020] The technical effects of the method revealed in the achievements include increasing the utility and performance of hyperspectral signature matching, especially when tracking and object detection methods are used in conjunction with the method. The method reduces the number of hours of work required to observe a tracked target. Also, the method improves the association and autonomous identification of new objects, related to those already tracked, prevents loss of tracking and extends autonomous tracking, removing the need for instant review by the human operator. This technique can be used in any system that generates images composed of the matrices of spectral cubes.
    Petition 870160036329, of 7/14/2016, p. 22/39
    8/23 [021] Figure 1 is a diagrammatic view of a method 10 for tracking and determining a probability of detecting objects observed in HSI according to a first embodiment of the invention. The remotely detected HSI that can include single images or a hyperspectral video transmission can be inserted in 12 to a processor capable of processing the HSI. The processor receives hyperspectral data at 12 and processes the data set within a set of hyperspectral image frames at 14 by performing a succession of known image processing steps that may include, but are not limited to, noise filtering, corner detection, image registration, homography and frame-by-frame alignment. The processor can then select candidate targets using search algorithms in 16 from the objects tracked in the hyperspectral image frames, where the candidate targets and the tracked objects are the sets of pixels that can represent the hyperspectral image of a real-world object of interest. For example, in an HSI collection system, that is, designed to search for moving targets, candidate targets can be moving objects. In this example, the processor can perform a computational search for the minimum discriminating characteristics that identify the moving objects in the HSI. In another example, a user of an HSI collection system manually selects pixels on a display and identifies the corresponding signatures for further analysis.
    [022] The processor can then track the candidate targets selected in 18 from the frame to the HSI frame. The processor can compare the selected candidate targets in 20 to make reference to the target models of the known targets stored in a model database in 28 where reference target models are the sets of pixels
    Petition 870160036329, of 7/14/2016, p. 23/39
    9/23 that may have been previously established to represent the hyperspectral image of a real-world object of interest.
    [023] In 22, the processor can make a match comparison. If a selected candidate target matches a reference target model from the model database at 28, the processor at 24 can then determine a degree of correspondence between the selected candidate target and a target reference model and a probability that the selected candidate target has been detected. If the selected candidate target does not match a model, the processor can even then consider the selected candidate target to be a new reference target model at 30 or discard the same at 32. If the selected candidate target is considered a new model at 30 , the processor can then add data from the new model to the target model database at 28.
    [024] After determining the degree of matching and the probability of detection at 24, the processor can compare the probability to a threshold at 26. If the probability exceeds a threshold, the processor can take action at 34. Otherwise, the processor can continue to track the candidate target selected at 18.
    [025] After the target specific reference models are identified from the target reference model database at 28 and compared at 20 with the candidate targets, the processor can calculate the degree of match and the probability of detection at 24. The degree of match and probability of detection can ask for the likelihood that the selected candidate target is a match for a specific reference target model by first comparing 24 of the top spectral signatures that appear on the selected candidate target with the top spectral signatures that define the target reference model, and then combine them spatially.
    Petition 870160036329, of 7/14/2016, p. 24/39
    10/23 [026] The processor that computes the method of determining the degree of correspondence and probability of detection in 24 can first determine the set of main signatures that appear in both the selected candidate target and the target reference model. Then, the processor can calculate the distribution of those main signatures based on the number of pixels in both the selected candidate target and the reference target model. To do this, the first step is to determine the set of signatures in the target reference model that cover a certain percentage of the pixels in the target reference model and determine the percentage of each of the signatures in the target reference model. The processor computing the method at 24 can then determine the distribution of signatures for a selected candidate target. If the pixel distribution in each signature is similar to the signature distribution in the target reference model, the processor that computes the method can then calculate the degree of correspondence for each of the signatures considering the maximum and minimum difference between the pixels of similar signature. The processor that computes the similarity between the hyperspectral pixel distributions can employ one or more similarity measures for computing. Similarity measures can include SAM, SID, ZMDA or Bhattacharyya distance. The processor may employ other similarity measures depending on the deployment.
    [027] Let Si = {si, S2, · · ·, Sp} be the set of signatures on a target and let Xj be a pixel at location ij in two dimensional spatial representations of a hyperspectral frame. The pixel Xj is offset from a matrix of subpixels so that the pixel xij has a set of values Xbi, Xb2, · · ·, xbq where q is the number of spectral bands in the hyperspectral images. Therefore, each pixel contains a subpixel value associated with each spectral band for the spatial location revealed by the pixel.
    Petition 870160036329, of 7/14/2016, p. 25/39
    11/23 [028] A selected candidate target that is referred to in the document for brevity as object O, which spatially combines the reference model target that is referred to in the document for brevity as the target 7 can also spatially match target T with C confidence if the set of main signatures R% on target Disappear in a similar proportion λ on object 0 /. The goal is to match the object and the target spatially and spectrally, that is, the shapes and signatures of the object and target are similar.
    [029] Let Λ /, - be the number of pixels in object O, and nu, n-, 2 ,, laugh with r <p that defines the cardinality or the size of the pixel sets in object O, · which has signatures similar Si, s 2 ,, s r . The processor that computes the method in 24 considers two objects O, and Oj a spectral match if the top R% of the spectral signatures on object O, correspond to the main signatures R% of the object Oj. The two objects 0 / and Oj correspond in λ precisely if for all selected numbers of main signatures of object 0, and Oj denoted as {nn, n-, 2 ,, and {ημ, rij2,, rij r } respectively:
    «Ii
    Λ / ; rç / i <2 <λ [030] The degree of correspondence for each signature / can be defined as:
    ijjfOpOj) = 1 - niüLí, | x ; - XjjJ - min; | x ;! -
    Petition 870160036329, of 7/14/2016, p. 26/39
    12/23 [031] The method can use other definitions for the degree of correspondence for each signature, /. Any definition to determine the degree of correspondence at 24 needs to fit the known mathematical definition of a fuzzy measure.
    [032] Finally, the processor that computes the method in 24 can calculate a detection probability based on the similarity between the signature set in the model and the signature set in the object. Considering Λ // Ο number of pixels in object O, and Λ / y the number of pixels in object Oj, the processor can calculate the detection probability at 24 based on the degree of correspondence and the number of pixels that match each signature. The processor can calculate the probability of detection by normalizing the degree of correspondence in relation to the number of pixels of the object to determine a level of confidence related to how close the image of the selected candidate target object is to match the target reference model of hyperspectral image. The detection probability, referred to as TM, is computed as: y [033] Where the Ni / s number of pixels in Ο / λ corresponds to the signature s.
    [034] At 26, the detection probability or TTW for a candidate target object selected as a match for a target model can be compared to a threshold. As shown in 26, the processor can calculate the TM - 1 and compare the threshold, ε. If the amount of TM - 1 exceeds the threshold, ε, the processor can take action at 34. Otherwise, the processor can continue to track the candidate target selected at 18. The threshold value, ε, can be selected based on in the threshold implementation of the correspondence algorithm in 22, the search algorithm in 16 and the information belong to the specific candidate target and target reference model in
    Petition 870160036329, of 7/14/2016, p. 27/39
    13/23 database in 28 as the speed of the object calculated in the HSI situation.
    [035] Different levels of confidence are defined based on the value of TM. For example, in one case, if the TM is less than 0.35 the confidence level will be much lower; if the TM is between 0.35 and 0.60, the confidence level will be lower, if the TM is between 0.60 and 0.75, the confidence level will be average; if the TM is between 0.75 and 0.85, the confidence level will be medium to high; and if the TM is greater than 0.85, the confidence level will be high. As the likelihood of a match becomes more likely, a result display can iterate through a color sequence mapped to those TM levels to distinguish targets detected with a high level of confidence from targets detected with a low level of confidence . The pixels in an image of a target detected with a high level of confidence can, for example, all be red in a viewfinder. Other thresholds, confidence levels and display schemes can be used depending on the deployment.
    [036] When the processor receives data at 12 and processes it in a set of hyperspectral frames at 14, the processor can then select candidate targets at 16 from the hyperspectral frames. The processor can select and use a search algorithm for hyperspectral data to select candidate targets in 16. The size of the hyperspectral data may have been reduced through dimensionally known reduction techniques, which include, but are not limited to, analysis of main components, resource extraction and entropy measurements. Figure 2 is a schematic view of a method 100 for selecting a search algorithm for hyperspectral data according to an embodiment of the invention. Selecting a search algorithm for hyperspectral data, a processor that computes the method in 100 can initially save the characteristics of a hyperspectral picture for a database in
    Petition 870160036329, of 7/14/2016, p. 28/39
    14/23
    110. Therefore, the processor can evaluate a characteristic of the hyperspectral picture in 112. If the processor evaluates that the characteristic in 112 is of importance for the hyperspectral picture, processors can apply a search algorithm for the data in 116 to distinguish the targets board candidates. If the processor evaluates that the characteristic in 112 is not significant for the hyperspectral picture, the processor can evaluate a second characteristic in 114. If the processor evaluates that the second characteristic in 114 is of importance for the hyperspectral picture, the processor can apply a second spectral search algorithm in 120 for the data to distinguish the candidate targets of the frame. If the processor evaluates that the second characteristic in 114 is not significant for the frame, the processor can evaluate a third characteristic in 118. If the processor evaluates that the third characteristic in 118 is of importance for the hyperspectral picture, the processor can apply a third search algorithm in 122 for the data to distinguish candidate targets from the hyperspectral picture. If the processor evaluates that the third characteristic at 118 is not significant for the hyperspectral picture, the processor can apply a fault search algorithm 124 to the data.
    [037] Initially, the processor can determine characteristics of a hyperspectral frame at 110. The processor can save the characteristics of the hyperspectral frame at 110 so that they are available for further processing when selecting a search algorithm. Example features may include an estimate of the variability of the illumination of the imagined situation, the variability of pixels with similar signatures and the number of distinct signatures in the target reference model. Other characteristics of the hyperspectral picture can be considered and these examples may not be considered limiting.
    Petition 870160036329, of 7/14/2016, p. 29/39
    15/23 [038] Based on an evaluation of the first characteristic in 112 of the hyperspectral picture, the processor can apply a search algorithm that has been verified to work well with the hyperspectral data defined by that first characteristic in 116. If the evaluation of the first feature in 112 of the hyperspectral frame does not indicate that the first search algorithm will work well with the hyperspectral frame, the processor can access the saved frame characteristics from 110 for an evaluation of a second frame characteristic in 114. In for example, the first characteristic may be the variability of the illumination of the imagined situation of the hyperspectral picture. The processor can access the characteristics of the hyperspectral picture to determine the variability of the illumination of the imagined situation. The processor can make a decision to determine whether the variability is high or low. The processor can use other frame characteristics as a first frame characteristic depending on the deployment.
    [039] If the first hyperspectral picture characteristic is evaluated to be of importance, the processor can use a first search algorithm at 116 to process the hyperspectral picture and its candidate targets. In this example, if the processor calculates the high lighting variability of the imagined situation, a search algorithm based on SAM can process the imagined situation for the best results. The method can use other search algorithms based on the classification methods that include, but are not limited to the SID, the Mahalanobis distance, the ZMDA and the Bhattacharyya distance, depending on the implantation.
    [040] Based on an evaluation of the second characteristic in 114 of the hyperspectral picture, the processor can apply a search algorithm, that is, known to work well with the hyperspectral data defined by that second characteristic in 120. If the evaluation of the second 870160036329, of 07/14/2016, p. 30/39
    16/23 of the characteristic in 114 of the hyperspectral frame does not indicate that the second search algorithm will work well with the hyperspectral frame, the processor can access the saved frame characteristics from 110 for an evaluation of a third frame characteristic in 118 In one example, the second characteristic may be the variability of pixels with similar signatures. The processor can access the characteristics of the hyperspectral picture to determine the variability of pixels with similar signatures. The processor can make a decision to determine whether the variability is high or low. The processor may use other frame characteristics as a second frame characteristic depending on the deployment.
    [041] If the second hyperspectral picture characteristic is evaluated to be of importance, the processor can use a second search algorithm at 120 to process the hyperspectral picture and its candidate targets. In this example, if the processor calculates the high variability of pixels with similar signatures, a search algorithm based on SID can process the situation imagined for the best results. The method can use other search algorithms based on similarity or distance measurements, which include, but are not limited to SAM, the Mahalanobis distance, the ZMDA and the Bhattacharyya distance, depending on the implantation.
    [042] Based on an evaluation of the third characteristic in 118 of the hyperspectral picture, the processor can apply a search algorithm, that is, known to work well with the hyperspectral data defined by that third characteristic in 122. If the evaluation of the third characteristic in 118 of the hyperspectral picture does not indicate that the third search algorithm will work well with the hyperspectral picture, the processor can apply a fault search algorithm in 124 to process the hyperspectral picture. In one example, the third characteristic may be the number of distinct signatures in the target reference model. The processor can access Petition 870160036329, from 07/14/2016, p. 31/39
    17/23 the characteristics of the hyperspectral picture that previously includes the targets tracked and that correspond to the target model of references to determine the number of distinct signatures in the target model of reference. The processor can make a decision to determine whether the number of distinct signatures on the target reference model is high or low. The processor can use other frame characteristics as a third frame characteristic depending on the deployment.
    [043] If the third hyperspectral picture characteristic is evaluated to be significant, the processor can use a third search algorithm at 122 to process the hyperspectral picture and its candidate targets. In this example, if the processor calculates a high number of distinct signatures on the target reference model, a Mahalanobis distance-based search algorithm can process the situation imagined for the best results. The method can use other search algorithms based on similarity or distance measurements, which include, but are not limited to SAM, SID, ZMDA and Bhattacharyya distance, depending on the deployment.
    [044] After the frame characteristics are exhausted, the processor can use a fault finding algorithm at 124 to process the hyperspectral frame and its candidate targets. The fault-finding algorithm can be based on any SAM, SID, Mahalanobis distance, ZMDA and Bhattacharyya distance. The method can use another search algorithm as a failure search algorithm, depending on the implementation.
    [045] The 100 method can implement additional steps using other framework characteristics and their assessments. Frame characteristics can be chained in series in the decision steps that follow the decision steps previously revealed in 112, 114, and 118. Also, the processor can evaluate multiple frame characteristics to determine
    Petition 870160036329, of 7/14/2016, p. 32/39
    18/23 if a particular search algorithm is optimally developed to process the hyperspectral picture.
    [046] The method at 100 can implement additional search algorithms. For example, the processor can execute multiple search algorithms simultaneously in the hyperspectral frame. The processor can then aggregate the results using methodologies that make a multi-criteria decision based on the simultaneous processing of multiple search algorithms within a single result.
    [047] Figure 3 is a schematic view of a 200 method to select a tolerance for a search algorithm. When processing the hyperspectral picture with a search algorithm selected in 116, 120, 122 and 124 in Figure 2, the parameters or tolerances of the given algorithm can be initially set to a failure value or values in 210. The search algorithm it can then process the data from the hyperspectral frame along with the fault tolerances at 212. The selected search algorithm can compute the number of hyperspectral pixels to the hyperspectral frame that are determined to match the candidate targets of the hyperspectral frame to the target model in 216. If some hyperspectral pixels also match the candidate targets of the hyperspectral frame to the target reference model, the processor can yield tolerances to the search algorithm selected in 218 and then the search algorithm can then process the frame hyperspectral again at 212 with modified tolerances. If some hyperspectral pixels also match the candidate targets of the hyperspectral frame to the target reference model, the processor can reduce the tolerances for the search algorithm selected in 214 and then the search algorithm can then process the hyperspectral frame again in 212 with modified tolerances. If an acceptable number of hyperspectral pixels
    Petition 870160036329, of 7/14/2016, p. 33/39
    19/23 match, the processor can save the location and signatures of the hyperspectral matching pixels at 220.
    [048] The processor can repeat the steps of modifying the search algorithm tolerances in 214 and 218 followed by processing the hyperspectral frame with the search algorithm selected in 212 until the number of pixels matching in 216 is within the limits acceptable.
    [049] The method at 10 in Figure 1 for tracking and determining a probability of detection of objects observed in HSI according to a first embodiment of the invention can guide an action at 34 in Figure 1 based on the probability of detection for a candidate target that exceeds a threshold of 26 in Figure 1 based on the analysis of the spatial and spectral parameters of the candidate target relative to the models known in the reference target model database at 28. At that point, each candidate target can have a unique identifier associated with its . If the processor computing the 10 method in Figure 1 detects deviation in a candidate target based on changes in its spatial and spectral characteristics, the processor can then automatically mark the deviation as a significant event in that target's life cycle. The processor can then assign an identifier to identify the diverted target as a new object. The processor can aggregate all target events within a reviewable timeline, in which a human operator has the ability to assess and potentially correct the processor's choice of associating existing or new identifiers with tracked objects.
    [050] The processor that computes the method at 10 in Figure 1 can create an entry in the target model database at 28 in Figure 1 with descriptions of both spatial and hyperspectral information and the characteristics of the candidate target at the target selection point at 16 in Figure 1.
    Petition 870160036329, of 7/14/2016, p. 34/39
    20/23
    In addition to spatial and hyperspectral information, the model target database at 28 in Figure 1 can also store information about time as the processor tracks the candidate target in the HSI. If the processor detects a deviation in spatial or spectral parameters at 20 in Figure 1 used to track a candidate target, the processor can store information in the database at 28 in Figure 1 which classifies the changes as an event that can be used for the future review. In addition, the processor can even associate the same or a new unique identifier for the new object whose defined parameters are substantially different than the original target. The processor can base the decision to assign an event on a calculated confidence measurement to determine a significant deviation from the established parameters. The confidence measurement can use parameters defined in the spatial, spectral or both domains to be robust for the capture errors in the spatial and hyperspectral information.
    [051] There are many situations in which the parameters of a candidate target can deviate significantly from its previously established parameters and trigger an event. Such situations may include; a tracked object becomes obstructed by another object; a tracked object is divided into multiple separate objects; a tracked object significantly changes its spectral characteristics, such as color, contrast or brightness by crossing into an area covered by shadow. Other situations exist and these need not be considered limiting. If the processor cannot associate with a candidate target before and after such an event, the processor can associate the same identifier used for the candidate target before the event to one or more new candidate targets after the event, which removes the possibility of loss or wrongly strike the target.
    Petition 870160036329, of 7/14/2016, p. 35/39
    21/23 [052] Figures 4a and 4b demonstrate this exemplary situation. Figure 4a shows an exemplary situation at 300 in which the method for tracking and determining a probability of detecting objects observed in HSI according to an embodiment of the invention has detected and tracked two vehicles 310, 312 traveling on a path. The processor that implements the method at 10 in Figure 1 processes the hyperspectral data received at 12 in Figure 1 in a sequence of hyperspectral frames in 14 in Figure 1 to select candidate targets in 16 in Figure 1. Under a comparison of candidate targets in 20 in Figure 1 with models in the reference model target database in 28 in Figure 1, the calculations resulting from the degree of correspondence and probability of detection in 24 in Figure 1 are significant and triggering action in 34 in Figure 1. The processor designates the each car 310, 312 a target identifier that can be stored in the target model reference database at 28 in Figure 1.
    [053] Figure 4b shows a situation where the method for tracking and determining a probability of detecting the objects observed in HSI according to an embodiment of the invention has detected changes in the scanned objects. Figure 4b demonstrates an event whereby one of the candidate targets previously identified, a car 310 in Figure 4A travels under the shade 318 of a tree 324 thereby significantly changing the spectral characteristics of the tracked car 310 in Figure 4A. In addition, a second similar car is now traveling close to the previously tracked car 310 in Figure 4A. The processor computing the method as 10 in Figure 1 can now distinguish with low confidence that car 314 or 316 is the previously identified and tracked car 310 in Figure 4A. The processor can take action at 34 in Figure 1 to identify both cars 314 and 316 with identifiers that can be associated with the
    Petition 870160036329, of 7/14/2016, p. 36/39
    22/23 identifier of the previously tracked car and record the time of the event within the database at 28 in Figure 1 for both objects.
    [054] The second of the previously identified candidate targets, a car 312 in Figure 4A parks in a parking lot and a passenger 322 exits the vehicle now parked 320. The processor can detect and identify an event by which the original object of car 312 in the Figure 4A were separated into two objects tracked separately. The processor can take action like 34 in Figure 1 to identify both car 320 and person 322 with identifiers that can be associated with car 312 in Figure 4A before the event and record the time of the event within the database at 28 in Figure 1.
    [055] One benefit of storing event information and creating identifiers that can be associated with events is that a system operator can recall the event history of any target associated with any target identifier. The operator can then analyze all objects with that identifier or associated identifiers that are being tracked along with the object's history for review. The event history can include all data related to all events where the system's alternate identifier for the tracked objects. Additionally, the operator can manually correct the system if the identifier that was associated with a target or targets in an event is incorrect.
    [056] This written description uses examples to reveal the invention, which includes the best mode, and also to enable any person skilled in the art to practice the invention, including producing and using any devices or systems and carrying out any built-in methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have
    Petition 870160036329, of 7/14/2016, p. 37/39
    23/23 structural elements that do not differ from the literal language of the claims or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.
    Petition 870160036329, of 7/14/2016, p. 38/39
    1/1
    权利要求:
    Claims (5)
    [1]
    Claims
    1. METHOD (10) OF IDENTIFYING A TRACKED OBJECT, characterized by understanding the steps:
    - tracking an object with the use of an imaging sensor, in which the tracked object has a known database of spatial and hyperspectral information;
    - associate an identifier to the tracked object;
    - select at least one parameter associated with the spatial or hyperspectral information of the tracked object;
    - detecting a deviation in at least one selected parameter;
    - compare (20) the deviation with the database (28); and
    - if the deviation exceeds a predetermined threshold, assign a new identifier to the tracked object, and, if the deviation does not exceed the predetermined threshold, continue tracking the tracked object.
    [2]
    2. METHOD (10), according to claim 1 characterized by the fact that it comprises the stage of updating the database with a change in at least one parameter.
    [3]
    3. METHOD (10), according to claim 1, characterized by the fact that it comprises the step of identifying the step of detecting a deviation in at least one parameter selected as an event.
    [4]
    4. METHOD (10), according to claim 1, characterized by the fact that the at least one parameter includes one among a color, contrast, brightness, shape, speed, direction, size and location.
    [5]
    5. METHOD (10), according to claim 1, characterized by the fact that the step of comparing the deviation with the database is performed with one of the Spectral Angle Mapping (SAM), Spectral Information Divergence (SID) , Differential Angle Equal to Zero (ZMDA), Mahalanobis distance, Bhattacharyya distance.
    Petition 870160036329, of 7/14/2016, p. 15/39
    1/5
    类似技术:
    公开号 | 公开日 | 专利标题
    BR102013020974A2|2018-02-14|METHOD OF IDENTIFYING A TRACKED OBJECT
    EP2698767B1|2017-11-08|Method of selecting an algorithm for use in prcessing hyperspectral data
    US10614310B2|2020-04-07|Behavior recognition
    CN103914702B|2017-08-25|System and method for improving the object detection performance in video
    BR102013019507A2|2014-07-08|METHOD FOR ASSESSING THE CONFIDENCE OF EQUIVALENT SIGNATURES OF A HYPERSPECTRAL IMAGE AT LEAST ONE TRACKED OBJECT
    Granström et al.2011|Learning to close loops from range data
    Zaki et al.2013|A framework for automated road-users classification using movement trajectories
    Rios-Cabrera et al.2012|Efficient multi-camera vehicle detection, tracking, and identification in a tunnel surveillance application
    BR102013024785B1|2021-04-06|image processing system and method for tracking moving objects in a sequence of images
    CN105701448B|2019-08-09|Three-dimensional face point cloud nose detection method and the data processing equipment for applying it
    CN106327526B|2020-07-07|Image target tracking method and system
    Jedrasiak et al.2014|SETh: the method for long-term object tracking
    Granström et al.2010|Learning to close the loop from 3D point clouds
    Elseberg et al.2011|Full wave analysis in 3D laser scans for vegetation detection in urban environments
    Castañeda et al.2011|Non-overlapping multi-camera detection and tracking of vehicles in tunnel surveillance
    Elassal et al.2016|Unsupervised crowd counting
    Khanloo et al.2010|Max-margin offline pedestrian tracking with multiple cues
    Michael et al.2016|Fast change detection for camera-based surveillance systems
    Sykas et al.2012|Endmember labeling and Spectral Library building and updating based on hyperspectral images
    BR102013015147A2|2015-08-04|Method for selecting an algorithm for use in hyperspectral data processing
    Kupyn et al.2019|Fast and efficient model for real-time tiger detection in the wild
    Javanmardi et al.2019|Automated traffic sign and light pole detection in mobile LiDAR scanning data
    Bouachir et al.2013|Visual face tracking: A coarse-to-fine target state estimation
    US9256786B2|2016-02-09|Method of identification from a spatial and spectral object model
    Cheng et al.2014|Video-based automatic transit vehicle ingress/egress counting using trajectory clustering
    同族专利:
    公开号 | 公开日
    CN103810226B|2018-11-09|
    CA2823369A1|2014-02-17|
    US9122929B2|2015-09-01|
    JP2014038620A|2014-02-27|
    EP2698740A3|2016-06-01|
    CN103810226A|2014-05-21|
    US20140050352A1|2014-02-20|
    JP6411718B2|2018-10-24|
    EP2698740A2|2014-02-19|
    CA2823369C|2019-12-17|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US5559496A|1993-05-19|1996-09-24|Dubats; William C.|Remote patrol system|
    US6985172B1|1995-12-01|2006-01-10|Southwest Research Institute|Model-based incident detection system with motion classification|
    AT454789T|2002-11-12|2010-01-15|Intellivid Corp|METHOD AND SYSTEM FOR TRACKING AND BEHAVIORAL MONITORING OF MULTIPLE OBJECTS THROUGH SEVERAL VISIBILITIES|
    US6967612B1|2004-10-22|2005-11-22|Gorman John D|System and method for standoff detection of human carried explosives|
    US9103714B2|2009-10-06|2015-08-11|Chemimage Corporation|System and methods for explosives detection using SWIR|
    US20070224694A1|2006-02-10|2007-09-27|Puchalski Daniel M|Method and system for hyperspectral detection of animal diseases|
    US7796833B2|2006-02-16|2010-09-14|Cet, Llc|Method for spectral data classification and detection in diverse lighting conditions|
    US20080243439A1|2007-03-28|2008-10-02|Runkle Paul R|Sensor exploration and management through adaptive sensing framework|
    DE102009020709A1|2009-05-11|2010-11-18|Basso, Gertrud|Method for monitoring and analyzing territory and states of air and vegetation in forest land, involves storing information and operator decisions in respective database for analysis calculation of images|
    US8203114B2|2009-05-14|2012-06-19|Raytheon Company|Adaptive spatial-spectral processing |
    JP5285575B2|2009-11-04|2013-09-11|日本放送協会|Human behavior determination device and program thereof|
    IL204089A|2010-02-21|2014-06-30|Elbit Systems Ltd|Method and system for detection and tracking employing multi-view multi-spectral imaging|
    US8285079B2|2010-03-19|2012-10-09|Sony Corporation|Method for highly accurate estimation of motion using phase correlation|
    US20120274775A1|2010-10-20|2012-11-01|Leonard Reiffel|Imager-based code-locating, reading and response methods and apparatus|
    CN102393913B|2011-10-31|2017-04-05|北京航空航天大学|A kind of Weak target precise tracking method based on spectral fingerprint feature|
    US8989501B2|2012-08-17|2015-03-24|Ge Aviation Systems Llc|Method of selecting an algorithm for use in processing hyperspectral data|US9430846B2|2013-04-19|2016-08-30|Ge Aviation Systems Llc|Method of tracking objects using hyperspectral imagery|
    US9256786B2|2013-10-15|2016-02-09|Ge Aviation Systems Llc|Method of identification from a spatial and spectral object model|
    WO2015086295A1|2013-12-10|2015-06-18|Bae Systems Plc|Data processing method|
    AU2014361105A1|2013-12-10|2016-06-30|Bae Systems Plc|Data processing method and system|
    US10890648B2|2014-10-24|2021-01-12|Texas Instruments Incorporated|Method and apparatus for generating alignment matrix for camera-radar system|
    US11205270B1|2015-03-25|2021-12-21|Amazon Technologies, Inc.|Collecting user pattern descriptors for use in tracking a movement of a user within a materials handling facility|
    CN105043544B|2015-05-05|2017-06-20|中山大学|A kind of image spectrum tracking and system|
    EP3317624B1|2015-07-05|2019-08-07|The Whollysee Ltd.|Optical identification and characterization system and tags|
    CN106776641B|2015-11-24|2020-09-08|华为技术有限公司|Data processing method and device|
    WO2017147792A1|2016-03-01|2017-09-08|SZ DJI Technology Co., Ltd.|Methods and systems for target tracking|
    SE542124C2|2016-06-17|2020-02-25|Irisity Ab Publ|A monitoring system for security technology|
    CN108510519B|2017-02-28|2021-12-14|哈尔滨工业大学|Method for extracting characteristic spectrum of dynamic target in tracking process|
    US10657422B2|2017-04-20|2020-05-19|The Boeing Company|Methods and systems for hyper-spectral systems|
    CN107054654A|2017-05-09|2017-08-18|广东容祺智能科技有限公司|A kind of unmanned plane target tracking system and method|
    CN109215056A|2017-07-03|2019-01-15|昊翔电能运动科技有限公司|Target tracking method and device|
    CN109214243A|2017-07-03|2019-01-15|昊翔电能运动科技有限公司|Method for tracking target, device and unmanned plane|
    CN107679454A|2017-08-28|2018-02-09|深圳市盛路物联通讯技术有限公司|Method of environmental monitoring and relevant device|
    US20190096045A1|2017-09-28|2019-03-28|4Sense, Inc.|System and Method for Realizing Increased Granularity in Images of a Dataset|
    US10690525B2|2018-01-03|2020-06-23|General Electric Company|Systems and methods associated with unmanned aerial vehicle targeting accuracy|
    CN108225277A|2018-03-09|2018-06-29|深圳臻迪信息技术有限公司|Image acquiring method, vision positioning method, device, the unmanned plane of unmanned plane|
    CN109165628A|2018-09-12|2019-01-08|首都师范大学|Improve method, apparatus, electronic equipment and the storage medium of moving-target detection accuracy|
    CN112966134A|2021-03-26|2021-06-15|北京亿耘科技有限公司|Remote sensing image data processing method, system and storage medium|
    法律状态:
    2018-02-14| B03A| Publication of a patent application or of a certificate of addition of invention [chapter 3.1 patent gazette]|
    2018-11-21| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|
    2019-12-24| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|
    2020-06-23| B11B| Dismissal acc. art. 36, par 1 of ipl - no reply within 90 days to fullfil the necessary requirements|
    优先权:
    申请号 | 申请日 | 专利标题
    US13/588,568|2012-08-17|
    US13/588,568|US9122929B2|2012-08-17|2012-08-17|Method of identifying a tracked object for use in processing hyperspectral data|
    [返回顶部]